SmartTShirt Documentation
Overview
This project consists of two python-based applications. One (Service) running on a Raspberry PI Zero and the other (Server) on a server. The Server application lets the user control and program the Service through a web-based interface. The scope of the project is to help students learn the basics of the programming language python by programing the sensors attached to the Raspberry PI.
SOURCE CODE: https://github.com/mslyon/SmartTShirt
Architecture
Service
The service application is a python 3.x application running on Raspberry PI’s Raspbian operating system. The app itself has two main scopes. One is the control of the various sensors attached to the Raspberry PI and the other is the communication back and forth with the Server app through WebSockets with the help of the Tornado framework.
Tornado is a Python web framework and asynchronous networking library. By using non-blocking network I/O, Tornado can scale to tens of thousands of open connections, making it ideal for long polling, WebSockets, and other applications that require a long-lived connection to each user.
The service was built to be very robust. It can detect faulty sensors and it prevents the user from sending too many consequent requests to the sensors which might otherwise render the device non-responsive.
Server
The server application is a python 3.x application running on the Django 2 web framework. It can be installed on both Linux and Windows servers and for a smaller number of users it can run from any modern desktop or laptop.
The Server app renders a web user interface that can be accessed from any modern browser. The Server app connects to the Service through WebSockets to display real time sensor data and it lets the user send data to the Service.
Other than giving access to the Raspberry PI’s sensors, the Server’s web user interface lets students create their own profiles and it lets teachers assign each device to the students.
The Server app can be run from the Local Area Network but it can also be installed to and accessed from a remote location like a webserver running on AWS with it’s own domain.
Installation
The project folder contains two main subfolders. One for the Server and one for the Service. Each folder has it’s own setup.sh file which can be run and it should install all the needed dependencies. The whole install process is also described in two README.md files (one for each folder).
There is a manual step in the installation of the Service for the Raspberry PI which involves configuring the I2C bus. The I2C is a very commonly used standard designed to allow one chip to talk to another. The other step involves manually configuring the touch sensor because of some incompatibility between the sensors drivers and the raspbian operating system. All the procedures are described in the README.md file in the Service subfolder.
The Server as well can be installed and configured automatically through the setup.sh script in its own folder, manually with the instructions from the README.md file in the Server subfolder or it can be installed and run from a docker container. The instructions to build and run a docker container are also included in the README.md file.
Using Internal IT infrastructure
In the below sections, Smart Tshirt is installed and used within intranet infrastructure.
Considering the scope of the project, it’s recommended doing a normal setup of the Server web app because it has less overhead than the docker counterpart at the moment.
The Server app can be run in a single user mode from the Raspberry PI itself. This doesn’t require a server to run the application but both Service and Server are running in a standalone mode from the Raspberry PI. To access the web user interface, the device has to be on some kind of local area network and has to have an IP address. The single user mode requires additional setup steps which can be as well found the README file.
Both Server and Service use the Supervisor service to start them on boot and keep them running. The Supervisor service can, if the app crashes, restart it automatically without any additional input from the user.
Installation steps:
Server:
Copy full folder structure to your server
Do ‘chmod +x setup.sh’ in the main folder
Do ‘./setup.sh’
Everything should be installed and already configured
Note: as there are various Linux/Windows OS distributions, some commands may not work in which case manual installation is needed (see README for full instructions)
Service:
Copy full folder structure to your service (Tshirt)
Do ‘chmod +x setup.sh’ in the main folder
Do ‘./setup.sh’
Everything should be installed and already configured
Using external organization IT infrastructure
When Smart TShirt cannot be used with existing internal infrastructure (e.g., street demonstration or demonstration done at another school) then the following is suggested:
1 small server that will support all PIs in the closed network (ideally this can be a laptop with Windows 10 installed and VmWare application running Linux)
On VmWare Linux, server will be installed
1 WiFi access point strong enough to support at least 150 connected devices (e.g., D-Link Systems Wireless AC1200, Linksys Business LAPN600, D-Link Air Premier-N Dual-Band PoE, etc.)
All PIs should connect to the WiFi access point.
Current running server solution
Currently server is running on Linux based server with NGINX which is doing proxy from the port 8080 to the port 80.
The nginx configuration is in the folder /etc/nginx.
To accesss the web server there are 2 ways:
Domain name: http://domainname.com
IP address: http://IP_ADDRESS
To access the admin page a default account should be created (see next section on admin account creation)
Features
Once both the Server and the Service are up and running, the user interface can be accessed by typing the IP address of the Server the port 8080 (for example 10.0.0.1:8080). This is what the app looks like when we access it for the first time:
At the bottom of the screen you can always see a colored bar with the current connection status for your assigned device.
Admin Profile
Upon initial software installation Admin account has to be created on the server with the following command in the src folder of the installation path:
python manage.py createsuperuser
To create additional admin accounts there are 2 methods that can be used:
Use the above command
Go through software interface (see next screens)
Go to admin web pages: http://smartshirt.smartfeld.ch/admin/
Click on Add
Click on Change
Select user and adding or unassign super rights
New admin user will be created
User Profiles
The first thing to do is either create a new account by signing up or log in if the user already has an account. All students have normal accounts while teaches have to be added manually to the superuser group.
To add a teacher to the superuser group, another superuser (the first one gets created during the setup phase) has to access the admin interface at 10.0.0.1:8080/admin, the select the desired teacher’s account and under Permissions, add the Superuser status by clicking the checkbox. Once logged in, teachers have an ‘Admin’ label near their username in the top left corner of the screen.
Both students and teachers can personalize their own profile pages by adding additional information about themselves and their own profile picture.
To create new users there are 2 methods:
User register by themselves by Clicking Sign up on the main page
And just enter all information and click on SIGN UP:
Or
Admin user can create users by visiting admin section of the website
Clicking on Add and then filling in all user information:
Navigation
The top of the screen contains the navigation bar with the links to all the part of the apps. Those are:
Home
Sensors
Laufschrift
Blockly
Teachers also have one additional link in the navigation bar called ‘Assign devices’ to assign all the devices to the students.
Sensors
Main screen
The main Sensors screen displays the realtime data from the user’s connected Raspberry Pi
The Temperature data are always displayed as soon as the device is connected. Since the Color Sensor LED can get hot over time, it has to be manually turned on and off when needed. When it’s turned on the other three fields will start to display the data in real time and the while LED light on the Raspberry PI device will turn itself off.
Graph
While the interface is receiving data, those can be temporarily saved by turning the ‘Record Graph Data’ functionality. This will save all the data in three separated arrays which are then stored in the browser’s localStorage. This data will be accessible until the user starts another recording or logs off from the current session. The saved data can be visualized by opening the graph page.
The data can be viewed as a Bar graph or a Line graph. Additionally, all the data are displayed as arrays below the graph. Each Bar (or point in case of a line graph) can display additional information by hovering over with the mouse cursor.
Laufschrift
This page can be used to quickly send text to the Raspberry PI’s LED board. The user can type a text, pick the brightness and color of the message. By clicking apply, the message gets displayed on the device
Blockly
The main part of the Server web app is the Blockly page. Blockly allows students to learn the basics of programming by placing visual blocks on the canvas which are translated in real time into python code.
Code interface
The main part of the Blockly interface it’s the Blockly canvas and the python generated code.
On the far left we have the toolbar divided into categories. The first category belongs to the Smart T-shirt and sensors functionality. The other categories are more general-purpose programming categories. By selecting an element from the left toolbar and dragging it over the canvas, blockly generates python code in real time. Each time a block gets added or changed, the code itself will immediately change.
Blocks can be deleted by either selecting them and hitting the delete key or by dropping them over the trashcan in the bottom right corner of the blockly canvas.
The Smartshirt blockly functionality it’s defined in the blockly.html template inside the Server folder. It consists of an XML element (blockly.html:103) which defines the markup and content of the toolbar rendered on the page and a various javascript functions (blockly.html:720) which define the behavior of each of those blocks. Additional blocks can be added at any moment.
Button bar
The button bar of the blockly interface resides below the blockly canvas. Once the blockly code has been created, students can decide what to do with it.
Load Code will open a modal view which will display previously saved blocks of code. Each line will show the block name, it’s description and which user saved it. This area has a twofold functionality. By checking the “Display only for current user” checkbox, it will show only the blocks that the current student has saved. By unchecking it (default behavior) it will display all the code saved in the database which makes it ideal for sharing code between students or between teachers and students.
Load code is sortable – by clicking on any of the header columns, it will sort the code. Also, it is possible to display code only for current user or display for current class.
Save Code will also open a modal view. It lets students save the current block of code they are working on. For each block the student can type the block’s name and its description. This way it will be easier to find it later on. The block code text area displays a minified xml version of the code that it’s going to be saved in the database.
Fav01-02-03 buttons allows the student to save three favorite functions (blocks of code). Those are immediately saved on the Raspberry PI device and can be executed via touch sensor even when the device is not on the network anymore. One touch runs function saved under Fav01 button, two touches for Fav02 and three for Fav03. There is a 5-6s delay between the touching of the sensor and running of the function which is a hardware limitation and cannot be avoided.
Send Code sends the current block of code to the device. Once clicked the buttons gets a disabled status to avoid students sending multiple time the same code. Once the code finishes the execution, the Service sends a ‘done’ message to the server which re-enables the button.
There are two ways to debug the code sent to the devices. One is to open the inspect panel in the browser and select the console (available on any modern browser). It will display the current code being sent to the device as it is being sent. The other is to open an ssh connection to the raspberry pi and check the messages that the device will display. The service prints out all the print functions as stdout and displays all the error messages as stderr both visible in the shell console.
Send to Class is available only for teachers or users of the superuser group. It allows the user to send the current opened block of code to many devices. Once clicked, it displays a list of available devices and the teaches has to check each device where the code should be sent. Once clicked the send button it will display a message with an array containing all the selected devices. The code gets sent to all the available devices immediately but due to hardware limitations and the nature and quality of wireless networks, the executed code might not run in the exact instant on all the devices. This too is a hardware limitation.
The last two buttons (Show and Hide code) allows the users to hide the python code part of the screen and thus visualizing the blockly canvas over the whole width of the screen.
Custom blocks
To add custom blockly code within existing server’s code, the following has to be done:
Change toolbar.js file – in this file the runnable logic is controlled
In StartfeldSmartTshirt > Server> src> static> site> js> toolbar.js
Code that will be added has to be added
Change blockly.html – this file is responsible for the View part
In StartfeldSmartTshirt > Server> src> templates> blockly.html
From line 109: <!-- Blockly XML toolbar markup -->
The view logic has to be added – part that is visible in Blockly
Example:
In blockly.html for Rainbow:
<category name="Rainbow" colour ="50">
<block type="init_strip"></block>
<block type="rainbow"></block>
<!-- <block type="color_wipe"></block> -->
<block type="theaterChaseRainbow"></block>
</category>
And in toolbar.js you the runnable logic is placed
// Rainbow ----------------
Blockly.Blocks['init_strip'] = {
init: function() {
this.jsonInit({
message0: 'Initialise strip',
nextStatement: null,
colour: 30,
tooltip:
'Import all from smartshirt, fire Adafruit_NeoPixel and strip.begin().'
})
}
}
Blockly.Python['init_strip'] = function(block) {
return [
'from smartshirt import *\nstrip = Adafruit_NeoPixel(LED_COUNT, LED_PIN, LED_FREQ_HZ, LED_DMA, LED_INVERT, LED_BRIGHTNESS, LED_CHANNEL, LED_STRIP)\nstrip.begin()\n'
]
}
Blockly.Blocks['rainbow'] = {
init: function() {
this.jsonInit({
message0: 'Rainbow',
previousStatement: null,
nextStatement: null,
colour: 30,
tooltip: 'Run rainbow()'
})
}
}
Blockly.Python['rainbow'] = function(block) {
return 'rainbow(strip)\n'
}
Classroom management
There are different ways on how to add a user to a classroom.
Through user’s profile:
Each user can go to his/her own profile and edit his/her current classroom
Classroom name should be entered into “Group or Class” and Update should then be clicked.
Through Admin interface
Go to Admin section and click on change
Click on user’s email address
Add user’s classroom
NOTE: Classroom name cannot start with a number – it has to start with a letter!
How to manage classrooms
Classroom management is done via Admin interface
By clicking on Blockly blocks it is possible to update, delete and copy existing classrooms as well as manage existing blocks
To assign existing blocks to another classroom Admin has to 1) select Blocks and 2) Select UPDATE CLASS OF SELECTED ROWS and then 3) enter new name of the class into ‘Studentclass’ field and click on GO.
Copy an existing classroom
To manage one classroom, Admin has to select an existing classroom under FILTER – selecting the classroom will display all blocks within that classroom.
To duplicate another classroom into a new classroom:
Select classroom as per previous step
Select all records from the classroom (all just some records)
Select ‘DUPLICATE SELECTED RECORD’
Click on GO
You will notice that a new classroom with the name ‘temp_class’ has appeared – now the next steps is to rename that temp class to the name that you want by following the steps described in the next section “Change the name of an existing classroom”
Change the name of an existing classroom
To change the name of an existing classroom, select all the records you want to put into the new classroom, select the option “update class of selected rows”, enter new name into Studentclass (in the below example it is MYNEWCLASSROOM’ and click on GO. Your new classroom will appear below the list of existing classrooms.
Activate or Deactivate Blocks
Number of activate blocks drives how many of them will be Loaded into Blockly. The best is to DEACTIVATE all blockly codes that will not be used.
Go to Admin Blockly Blocks and then select all the blocks that you want to activate to deactivate.
You can also select ALL BLOCKS present on the platform (Select all 471 blockly blocks).
Click on GO and blocks will be activate or deactivate.
You can also do the same par classroom.
Add blocks
Delete blocks
To delete blocks in Admin page, click on Blockly blocks and you can either enter into a block by clicking on it or you can select the block and from the dropdown menu select ‘Delete selected blocks’ and click on GO.
Turtle with Blockly
It is possible to run python turtle code. In the top menu, Turtle link should be clicked and then code can be executed by clicking on run.
Send to Students
It is possible to send blockly code from teacher to all students.
Button ‘Send to Students’ should be selected.
On student side, within the browser a new message will appear asking student whether to accept or not the new code.
If accepted the new code will be directly placed in student’s blockly window.
Connecting TShirts: Method 1 - Assign devices
As previously mentioned, this part of the web UI is only available for teachers or users who have the superuser status. This interface allows the pairing of user accounts and devices. This is made possible by running the zeroconf service on both the Server and the Service. The most important facet of Zeroconf is the ability to easily browse and advertise for available services. Zeroconf allows users to easily network one or many devices without performing any setup actions.
When adding new TShirts to the same physical network, Zeroconf can be an ideal way of automatically discovering all TShirts that are present in the network.
Once server is started, it will get automatic registrations from all available TShirtts. That is, TShirt will auto-register with the server by providing its unique identified (Device UUID) which the server will register in the SQL database.
The Assign Device button will open a list of available users who can be paired with the selected device. This creates a record in the database and the pairing won’t be changed until the device gets assigned to someone else or just gets unassigned.
The Show Device button makes the selected device blink by displaying it’s UUID. This way one device can be easily identified among others in class.
Sometimes a device can change its UUID and thus appearing twice on the list of available devices. Those ‘old duplicates’ can be deleted from the database with the Delete Device button. Also, if the server wants to force the TShirt to re-register then the device should be deleted with ‘delete device’
Limitations of the automated way of connecting to the server is that all Tshirts and server have to be on the same physical network – example all of the devices have to be on the same 192.168.1.X network.
Connecting TShirts: Method 2 - Assign devices
Another possibility to connect TShirts to the server which can be used especially in situations when the server is not on the same physical network (example: server is hosted on Amazon AWS cloud), is to configure TShirts with a physical IP Address of the server.
This is done on the Service side within the config.ini folder where [fixed_host]. In the below example, server is residing on the host 192.168.1.100.
Important to note is that on the server, port 8080 has to be open.
[fixed_host]
host = 192.168.1.100
port = 8080
To configure Method2, script is available on the Service – changeserver.sh – Script will ask the user for new IP address of the server. To come back to the Metho1, lines [fixed_host] have to be deleted from the file.
Accessing PIs without internet connection/wifi through Zeroconf and Remote Desktop
In case, IP address is not known, or WiFi is not enabled, another method that can be used to access Pis is through Zeroconf.
First thing to make sure is that each PI has a unique name (by default it is raspberrypi).
To change PIs hostname script or manual instructions should be used.
Change hostname (example: mypi1)
Once these steps are done PI will have a unique name
To connect to PI through Remote Desktop and Zeroconf on Windows 10 the following has to be setup:
Download and install Zeroconf/Bonjour service on Windows 10 https://support.apple.com/kb/DL999
Press windows button
Enter gpedit.msc and launch it (you have to be administrator)
Navigate to Computer Configuration > Administrative Templates > Network > DNS Client
Enable the "Turn off Multicast Name Resolution" policy (just double click it and select Enable)
Now, open Remote Desktop and enter inside:
mypi1.local
(if your PI name is different enter YOURNAME.local)
How to create and prepare Raspberry PI image
After creating PI image (e.g., installing all service related software), image can be extracted using a SD card reader and extraction software (e.g., Win 32 Disk Imager), the image size has to be changed so allow faster deployment on new PIs.
Click on the Read button. The card will then be backed up to your PC.
The following instructions have to be followed in order to reduce the PI image size:
Resizing Your Image
First, copy your backup image (called MyImage.img) to your hard drive or USB drive that you load to your Linux system.
We'll begin by getting the start point of the second partition on the uSD card. A Raspberry Pi image has two partitions, 1 Boot, 2 Where everything else is stored. To determine the start of the second partition enter the command below;
sudo fdisk -l /media/pi/1TBFREECOM/MyImage.img
Take note of the 'Start' of the second partition '.img2', we'll need it later. Here : 131072 (see pic 1 above, red circle).
Now we mount this partition using the following command (see pic 2 above);
sudo losetup /dev/loop0 /media/pi/1TBFREECOM/MyImage.img -o $((START*512))
Where START is the start of MyImage.img2 the second partition. ie. 131072. So in this case it will be;
sudo losetup /dev/loop0 /media/pi/1TBFREECOM/MyImage.img -o $((131072*512))
Note : Don't worry if you get a busy message, just enter the following and retry the above;
sudo losetup -d /dev/loop0
Now we resize the partition with 'GParted'. From a terminal window enter the following command (see pic 3 above);
sudo gparted /dev/loop0
This will launch the 'GParted' application (see pic 4 above).
From GParted do the following;
Select /dev/loop0 partition (pic 5 above),
Select Partition menu -> Resize/Move (pic 6 above),
Resize the partition by either dragging the slider left as indicated, or entering a value into the box marked 'New size (MiB)' (again as shown in pic 6). I found this to be a little trial and error and generally started with +500M and moved up in increments. ie Change the value of "New size (MiB)" to approx 500MB above "Minimum Size". In this case 2942 + 500MB = 3442MB etc.,
Click 'Resize/Move' to finish,
GParted will now indicate there is '1 operation pending' (as in pic 7 above),
Click Edit menu -> Apply All Operations,
Click 'Apply' to warning box (pic 8 above),
GParted will now process the request to resize (see pic 9),
If the resize is successful you will get a message: 'All operations successfully completed' (as in pic 10),
Now expand the 'Applying pending operations' dialogue shown in pic 10 until you locate the line in pic 11, marked 'resize2fs -p /dev/loop0 ...' and take a note of the value. Here : '3506176K'. This is important, so, take your time to get it right.
Note : If you get an error message at step 9 (as in pic 12), then click Ok, Close, and repeat from step 1 above, only this time add 500MB to your last resize. Repeat this until the size is accepted.
It is now safe to exit GParted. Press Ctrl+Q (^Q), or menu GParted -> Quit.
We will now remove the loopback device for the second partition and create a new loopback device for the whole image. Enter the following commands (see pic 13);
sudo losetup -d /dev/loop0
sudo losetup /dev/loop0 /media/pi/1TBFREECOM/MyImage.img
Next we need to edit the partition table to reflect the new smaller size. Begin by entering the following command;
sudo fdisk /dev/loop0
For the record 'fdisk' can be tricky to use, but in a 'nutshell' we are going to do the following (pressing the return/enter key after each character entry);
Enter 'd' then '2' to delete the table entry for the second partition,
Enter 'n' then 'p' then '2' to create a new second partition entry,
Enter the START sector number that you used earlier, as the start sector. In my example it was '131072',
Enter +NEWSIZE as the new size. Don't forget the '+' sign at the start. This is the new size that you noted down before exiting GParted. Here '3506176K',
Enter 'w' to write the new partition table and exit.
Don't worry about 'Re-reading the partition table ...' error message.
Picture 14 above details what needs to be entered (the important parts are ringed in red).
Once the partition table has been written we will read the partition table to determine the end point of the second partition by issuing the following command (see pic 15 above, the end point is ringed in red);
sudo fdisk -l /dev/loop0
Pay attention to end point of the second partition.
Here it is labeled as '/dev/loop0p2' and has a value of '7143423'
Next we remove the loopback device with the following command (see pic 15 above);
sudo losetup -d /dev/loop0
Finally, we trim any empty space with the truncate command the template for which is shown below, where END is the value obtained above. Namely : 7143423
truncate -s $(((END+1)*512)) /media/pi/1TBFREECOM/MyImage.img
Consequently to execute the full command enter the following (see pic 16 above);
truncate -s $(((7143423+1)*512)) /media/pi/1TBFREECOM/MyImage.img
And that's it. Your HDD will now contain an image file named 'MyImage.img' which has been successfully resized and can by written back onto a uSD card to restore your original image should the need arise.
IMPORTANT NOTES:
Python3 works on both Windows 10 and Linux so the whole could should work without any issue on both platforms
Code was tested on Windows 10, Ubuntu 16.10, Ubuntu 18.04 and Mint 18. For other versions slight adaptations might be needed to the installation scripts (e.g., apt-get is an Ubuntu command – on other seems command should be adapted to yum or similar)
For sensors to work, there is a possibility that the version of the Rapsberry needs to be reconfigured:
Enable I2C, see:
There is a possibility that touch sensor won't work, the following needs to be added
dtoverlay=i2c-bcm2708 to /boot/config.txt and reboot. See:
Finally, the command ‘sudo pip install Adafruit-MPR121’ needs to be executed
Last updated
